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(57) Abstract 



A digital radiologic image, notably a 
mammogram, is automatically processed by 
a computer to identify suspect masses. The 
identification is done by thresholding at least 
a region of interest of the image at, at 
least, 20 threshold levels determined from a 
histogram of the image to discriminate spots, 
and classifying the spots by size, shape and 
variance in intensity of the pixels comprising 
the spot. A processed mammogram having 
the suspect masses marked or enhanced is 
produced and displayed. 
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1 

Mass Detection in Digital X-ray Images Using Multiple Threshold Levels To Discriminate 
Spots. 

BACKGROUND OF THE INVENTION 

1 Field of the Invention 

The present invention relates generally to methods of and systems for computer 

aided diagnosis of radiologic images. In its more particular respects, it relates to 
S identification of spots that may correspond to suspect masses. The invention is particularly 

pertinent to computer-aided diagnosis of mammograms but is also useful with respect to 

other common radiologic images, for example, chest X-rays. 

7 Description of the Related Art 

A method of this general type for detection of suspect masses, nodules or 
10 tumors is known from U.S. Patent No. 5,212,637 and from U.S. Patent No. 4,907,156. 

Breast cancer is one of the primary causes of death for women in western 

societies. Because the vast majority of deaths due to cancer that originated in the breast 

could be prevented by early detection, national health organizations in the United States. 

recommend that all women over the age of 50 have regular screening mammograms. Similar 
15 recommendations exist in other countries. If these recommendations were followed by all 

women, the number of mammograms taken and read in the U.S. would overload the current 

capacity. 

Hie reading or interpretation of screening mammograms is an art that requires 
extensive experience and attentiveness to detail. The mammographer's primary sign for 

20 cancer is a moderately sized spot identified as an abnormal mass. Another sign is the 

presence of relatively small spots corresponding to minute deposits of calcium salts known as 
microcalcifications, which are arranged in clusters. Both the number of mammograms to be 
interpreted and the difficulty of identifying the aforementioned and other not easily 
recognizable signs of malignancy motivate developments in Computer-Aided Diagnosis of 

25 Mammograms (CADM) to at least automatically mark or enhance features of interest, 
including suspect areas, in images displayed on a monitor for interpretation by the 
mammographer. 

Digital mammograms suitable for computer-aided diagnosis may be obtained by 
scanning film taken by conventional X-ray mammography or by utilizing other X-ray 
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detector types that produce electronic image signals that may be directly digitized without the 
necessity of producing a film intermediate. These detector types include X-ray image 
intensifier/camera chain, photostimuable phosphor plate/laser readout (see U.S. Patent No. 
4,236,078), and selenium plate/electrometer readout technologies. Such technologies are 

5 progressing in their spatial resolution and contrast sensitivities achieved and the latter two, 
particularly, may soon find widespread use for mammographic applications. 

In U.S. Patent No. 5,212,637, a search for masses in different intensity ranges 
utilizes a calculated initial threshold value which threshold value is incremented no more than 
three times. "Blobs" produced by thresholding the mammogram at the initial or at an 

10 incremented threshold value, which correspond to regions having a sufficient prominence in 
intensity with respect to their immediate surround are classified as "potentially malignant" 
based on their size and shape, i.e. area, circularity, and eccentricity. 

In U.S. Patent No. 4,907,156, "islands" are produced by thresholding a 
difference image determined from a digital X-ray image. The threshold level is successively 

15 decreased to values producing a 1 % change in the area under a histogram of the pixel values 
in the difference image. In general, the islands grow as the threshold level is decreased. At 
each threshold level, the shape and size of the islands are analyzed. An island is classified 
as a nodule if it has either (1) an effective diameter of between 3 and 18 mm and a 
circularity of at least 0.85 for ten consecutive threshold levels, or (2) an effective diameter of 

20 between 9 and 18 mm and a circularity of at least 0.75 for four consecutive threshold levels. 

Such methods may yield false negative results by not detecting an abnormal spot 
that may only be discriminated within a limited range of intensity threshold values. Further, 
the aforementioned methods may yield an unacceptable number of false positive results, 
particularly when attempting to increase their sensitivity. 

25 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide a method and apparatus for 
aiding diagnosis of suspect spots in radiologic images which reduces the incidences of false 
negative results to substantially zero by utilizing a large number of relevant threshold levels 
30 while maintaining the incidence of false positive results to acceptable levels, e.g. on average, 
no more than one improperly marked spot per image. 

Briefly, these and other objects are satisfied by using a histogram of the gray 
levels of pixels in a radiologic image whose background has been removed from 
consideration to determine a relevant interval of gray levels, and thresholding the resulting 
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image at each gray level in the interval. Typically, in an 8 bit (256 gray level) image, the 
relevant interval contains at least 20 and typically more than 50 different gray levels. The 
image resulting from each thresholding operation is separately analyzed to determine 
connected regions or spots, and to classify which connected regions are suspect. At each 

5 gray level in the interval, pixels that belong to a suspect connected region are marked. This 
marking process is cumulative, since, after thresholding at each gray level in the interval, all 
pixels belonging to a suspect connected region in a binary image produced at any threshold 
level in the interval have been marked. 

The determination of which connected regions are suspect is done by forming 

10 measures of their respective size, shape and intensity variance or uniformity. Each 

connected region whose measures all meet predetermined criteria are designated as suspect. 
The inclusion of an intensity variance or uniformity criterion is because a typical suspect 
mass appears to have a relatively uniform intensity throughout its area. 

Thresholding at each gray level in a relatively large interval of relevant levels, 

15 tends to produce a 100% sensitivity in identifying suspect masses. The maintenance of a low 
rate of false positives is aided by the inclusion of the criterion requiring that the pixels of a 
connected region have low variance or high uniformity in intensity. This criterion, among 
other things, prevents substantially hollow connected regions from being designated as 
suspect. 

20 

BRIEF DESCRIPTION OF THE DRAWING 

Other objects, features and advantages of the invention will become apparent 
upon perusal of the following detailed description when taken in conjunction with the 
appended drawing, wherein: 
25 Figure 1 is a schematic diagram of a computer-aided system in accordance with 

the invention for taking and processing mammograms; 

Figure 2 is a flow chart indicating the processing performed by a computer in 
Figure 1; and 

Figure 3 is a histogram of the gray values of the pixels of a mammogram after 
30 background removal. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Referring first to Figure 1, there is shown a computer-aided mammography 
system 10, with its mammogram taking parts arranged for a cranio-caudal (CC) view, 
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including an X-ray source 12 directed to irradiate a breast 14 of a standing subject with an 
X-ray beam IS. The breast 14 is received and compressed between generally planar lower 
and upper members 16, 18, using a predetermined compression force or weight. Below 
lower member 16 is a two-dimensional X-ray detector means 20 for detecting within a 
5 rectangular field of pixels, the X-ray radiation passing through breast 14 and its immediate 
external surround. X-ray detector means 20 is alternatively a film or a photostimuable 
phosphor image plate received in a holder, or a selenium plate/electrometer readout detector. 
An X-ray image intensifier/camera chain is also a suitable detector means. The X-ray source 
12, plates 14 and 16 and detector means 20 may be rotated as a unit about transverse axis A 
10 to receive and irradiate breast 14 along any of the viewing directions labelled in Figure 1 as 
CC (cranio-caudal), LM or ML flatero-medial or medial-lateral) and OB (oblique). 

Whichever detector means 20 type is used, ultimately there is a two-dimensional 
array of digital pixels, representing the mammogram X-ray projection image, stored as an 
image file in a digital storage device 22 which may comprise a RAM, hard disk, magneto- 
15 optical disk, WORM drive, or other digital storage means. When film is used, it is 

developed and then scanned in a digitizer 24. Today, films may be digitized to 100 micron 
spatial resolution, yielding typical images ranging in size from 1672x2380 to 2344x3016 
pixels, each up to 12 bit intensity resolution. When a photostimuable plate is used, it is 
scanned by a laser in scanner 26 yielding a similar image size and typically 10 bit intensity 
20 resolution. Lastly, when a detector such as a selenium plate/electrometer readout device is 
utilized, it directly produces analog electrical signals that are converted to digital form by its 
digital to analog converter 28. 

The two-dimensional array of digital pixels stored in device 22, representing the 
mammogram, is processed by computer workstation 30 to mark or enhance features of 
25 interest in the mammogram, including any identified suspect masses or clusters of 

microcalcifications, and display the resultant processed mammogram on display device 32, 
such as a CRT monitor. As a preliminary step, the stored mammogram may be reduced in 
resolution, spatially by a suitable median filter, and/or in amplitude by truncation, to an 
image on the order of 500,000 to 2,500,000 pixels and 8-bit to 10-bit intensity resolution 
30 consistent with the spatial and gray scale resolution of the monitor. In the processing to 
mark or enhance features, the mammogram is segmented into foreground, corresponding to 
the breast, and background, corresponding to the external surround of the breast and the 
skinline is detected in the course of this segmentation. The segmentation allows background 
to be eliminated from the search for features of interest, such as masses or clusters of 
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microcalcifications, to be marked or enhanced. The segmentation may be performed by the 
method described in U.S. patent application serial number 08/175,805, filed December 30, 
1993, entitled "Automatic Segmentation and Skinline Detection in Digital Mammograms", 
which is assigned to the same assignee as the present application and is hereby incorporated 
5 herein by reference. The identification of suspect clusters of microcalcifications is described 
in U.S. Patent Application, Serial No. 08/003,071, filed January 11, 1993, entitled 
"Computer Detection of Microcalcifications in Mammograms", which is also assigned to the 
same assignee as the present invention. The latter application is also hereby incorporated 
herein by reference. 

10 Now referring to the flowchart shown in Figure 2, the identification of suspect . 

masses in a two-dimensional mammogram projection image will be described. The function 
of the boxes in the flowchart are given in Table I. It is assumed that as referred to 
heretofore, the original mammogram has been reduced in spatial resolution to about 250,000 
pixels (e.g. 480 X 520) and segmentation has already taken place by skinline detection. As a 

15 result, each pixel in the background has been removed from further consideration in the 
search for features of interest. Then, in step 34, a histogram of the gray values of the pixels 
in the foreground is calculated and a relevant interval of gray levels for thresholding is 
determined from the histogram. 

A typical histogram is shown in Figure 3 and is seen to be subdividable into an 

20 interval "a" from the smallest gray level S in the histogram to a gray level G, which interval 
corresponds to the skin, and a narrower interval "b" from level G to the largest gray level L 
in the histogram, which interval corresponds to the interior of the breast. Interval "a" has a 
substantially low number of pixels at each gray level while interval "b" has a relatively high 
peak with steep sides. The interval "b" is chosen as the relevant interval of gray levels for 

25 thresholding. Gray level G is chosen such that interval "b" is twice the interval "c" between 
gray level L and the gray level P at the peak of the histogram. 

In accordance with the invention, each gray level in interval "b" is used as a 
threshold. Typically, in a 256 gray level image interval B b" contains at least 20 twenty gray 
levels, and often more than 50. Conveniently, these gray levels are successively used as a 

30 threshold level in either smallest to largest, or largest to smallest, order. In any event, a 
current threshold level is set at the first gray level interval "b" in the order. 

Not in the flowchart of Figure 2, in step 36 the image is thresholded at the 
current threshold level to produce a binary image whose pixels having an intensity greater 
than or equal to the current threshold level are assigned the value one and whose pixels 



having an in tensity less than the current threshold level are assigned the value zero. 
Alternatively, the thresholded image may be a grey scale image obtained from the image 
input to step 36, by copying all pixel intensities that are not less than the current threshold 
level and setting the pixels having an intensity less than the current threshold level to the 
5 value zero. 

In step 38, spots referred to as "connected components" are extracted from the 

thresholded image. Each "connected component" is a set of pixels having non-zero values, 

in which any two pixels of the set are ultimately connected to each other via a run of 

adjacent pixels in the set. These sets are identified conveniently by the following phases: a) 
10 generating a line-Adjacency Graph (LAG), b) scanning the LAG to determine the number of 

different connected components (CCs), and c) again scanning the LAG to create a mask 

image and several summary arrays that define and describe each CC. 

The method to create an LAG in phase a) above is based on the description in 

the book "Algorithms for Graphics and Image Processing" by Pavlidis, Computer Science 
15 Press, 1982, pp. 1 16-120. It consists of for each line of the thresholded image, finding runs 

of adjacent non-zero valued pixels, comparing the position of the runs on the current and 

prior adjacent line, and recording any overlap. 

Although the LAG specifies which lines overlap, it does not define a connected 

component. Thus in phase b), each the record of overlapping runs is scanned to determine 
20 to which CC each run belongs. Along the way, the total number of connected components is 

computed. 

Once the set of CCs is known, then in phase c) a mask image and several data 
objects to define each CC are computed. The mask image is essentially the thresholded 
image in which all non-zero pixels contain the number of the CC to which they belong. The 
25 additional data objects include a vector containing the number of pixels in each CC, which is 
a measure of area, and an array defining a bounding box (minimum and maximum column 

and row) for each CC. 

After the extraction of connected components, in step 40 the variance Var of 
the intensities of the pixels in each CC is computed separately for each CC in accordance 
30 with the following equation: 

Var = -£(*r>0 2 
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where g; is the intensity of the ith pixel in the connected region, §i is the mean value of the 
intensities of the pixels in the connected region and n is the number of pixels in the 
connected region. Alternatively, a measure of uniformity might be used, for example, the 
inverse of the variance. 

5 After, the computation of variance, the connected regions are smoothed in step 

42 by erosion and then dilation. Preferably a structure element which is a 3 by 3 matrix of 
ones is used for each operation. These operations smooth sharp irregularities in the 
boundaries of the connected regions. Thereafter in step 44, an area measure A for each 
smoothed CC is computed, conveniently as the number of pixels therein. Then in step 46 a 

10 compactness measure Compact is computed for each CC as follows: 

P 2 

Compact = — 



where P is the perimeter of the connected region and A is its area. 
An eccentricity measure Ecc is formed in step 48 

as follows: 



15 Ecc = 



where r^ and r^ are the maximum and minimum distances between the crater of the 

connected area and the perimeter. 

Then in step 50, the aforementioned measures Compact, Ecc, Var and A are 

compared with threshold values T c , T E , T v and T A , respectively for each CC. If Compact, 
20 Ecc, and Var are less than or equal to T c , T E , and T v , respectively and A is greater than or 

equal to T A each pixel of the CC is marked as belonging to a suspect CC. After all CC's 

that were discriminated by the current threshold level are considered, then in step 52, it is 

tested whether there are more gray levels in the interval. If there are, then in step 54 the 

current threshold is changed to the next threshold level in order and step 26 is returned to 
25 where the original background removed image is thresholded at the new current threshold 

level. Steps 36 to 50 are repeated for each threshold level until it is determined at step 52 

that there are no more gray levels remaining in the interval. 

It should be appreciated that the marking of pixels as belonging to a suspect 

connected area is a cumulative operation. After, all threshold levels in the interval have 
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been applied, then the result 36 is suspect masses revealed from any of the threshold levels. 

Values of the thresholds used to test the measures A, Compact, Ecc and Var 
were obtained empirically from training with six images. The following values thus obtained 
have given good results with a 256 gray scale 480 by 520 pixel image, in which each pixel 
represents a square 0.4 mm on a side: 

T A = 130 pixels 

T c = 150 

T E = 2.7 

T v = 50 

In experiments run with respect to nine mammogram images, the automated 
procedure of the present invention successively marked all of the areas that had been marked 
by a radiologist as suspect, while marking one false positive in five of the nine images. 

It should now be appreciated that the objects of the present invention have been 
satisfied. While the present invention has been described in particular detail, numerous 
modifications are possible within the intended spirit and scope of the invention. 



Table I 



Legend to Figure 2 



Determine Interval of Gray Levels For Thresholding and Set Threshold Equal to 

Initial Gray Level in Interval 

Threshold Image 

Extract Connected Components 

Calculate Variance of Intensity Values For Each Connected Component 
Apply Erosion Then Dilation To Each Connected Component 
Calculate an Area Measure For Each Connected Component 
Calculate a Compactness Measure For Each Connected Component 
Calculate an Eccentricity Measure For Each Connected Component 
Mark Each Connected Component That Has Large Area and Small 
Compactness, Eccentricity, and Variance as Suspect Mass 
More Gray Levels in Interval? 
Set Threshold Equal to Next Gray Level in Interval 
Suspect Masses 
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CLAIMS: 



\ t A method of producing a computer-enhanced radiologic image comprising: 

irradiating a region of a body being examined with X-ray radiation in a 
predetermined viewing direction; 

receiving the X-ray radiation exiting the region from said viewing direction 
5 within a two-dimensional field; 

producing digital signals as a function of the X-ray radiation received, which 
digital signals correspond to an input two-dimensional array of digital pixels; 

in response to said signals, storing said input two-dimensional array of digital 
pixels in a digital memory means accessible to a computer; 
10 with said computer, processing the stored input two-dimensional array of digital 

pixels by thresholding at least a portion of the stored input array at, at least, 20 different 
threshold levels to discriminate spots, for each spot discriminated, forming a plurality of 
measures of said spot, including of its size and shape, and determining whether the formed 
plurality of measures fall within predetermined criteria, and producing an output two- 
15 dimensional array of digital pixels from the input array in which any spots whose formed 
measures meet said criteria at any threshold level are marked or enhanced, independently of 
whether or not corresponding spots meet said criteria at more than one threshold level; and 

displaying the output array of digital pixels as an image on a display means. 
2. The method as claimed in Claim 1, wherein said 

20 formed measures include one of the variance or uniformity of values of the digital pixels 
comprising said spot. 

3 a method of producing a computer-enhanced radiologic image comprising: 

receiving and storing in a digital memory means accessible to a computer an 
input two-dimensional array of digital pixels produced by a radiologic imaging apparatus; 
25 with said computer, processing the stored input two-dimensional array of digital pixels by 
thresholding at least a portion of the stored input array at, at least, 20 different threshold 
levels to discriminate spots, for each spot discriminated, forming a plurality of measures of 
said spot, including of its size and shape, and determining whether the formed plurality of 
measures fall within predetermined criteria, and producing an output two-dimensional array 
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of digital pixels from the input array in which any spots whose formed measures meet said 
criteria at any threshold level are marked or enhanced, independently of whether or not 
corresponding spots meet said criteria at more than one threshold level; and producing an 
output two-dimensional array of digital pixels from the input array in which any spots whose 
5 formed measures meet said criteria are marked or enhanced. 

4. The method as claimed in Claim 3, wherein said 

formed measures include one of the variance or uniformity of values of the digital pixels 
comprising said spot 

5. A system for producing a computer-enhanced radiologic image comprising: 
10 means including an X-ray source, for irradiating a region of a body being 

examined with X-ray radiation in a predetermined viewing direction; 

means for receiving the X-ray radiation exiting the region from said viewing 
direction within a two-dimensional field; 

means for producing digital signals as a function of the X-ray radiation 
IS received, which digital signals correspond to an input two-dimensional array of digital pixels; 

a computer; 

a digital memory means accessible to said computer; 

means for, in response to said signals, storing said input two-dimensional array 
of digital pixels in said digital memory means; 

20 wherein said computer is configured for processing the stored input two- 

dimensional array of digital pixels by thresholding at least a portion of the stored input array 
at a plurality of different threshold levels to discriminate spots, for each spot discriminated, 
forming measures of the size and shape of said spot and of the variance or uniformity in 
values of the digital pixels comprising said spot, and determining whether the formed 

25 measures fall within predetermined criteria, and producing an output two-dimensional array 
of digital pixels from the input array in which any spots whose formed measures meet said 
criteria are marked or enhanced; and 

display means for displaying the output array of digital pixels as an image. 

6. The system as claimed in Claim 5, wherein said 

30 formed measures include one of the variance or uniformity of values of the digital pixels 
comprising said spot. 

7. A system for producing a computer-enhanced radiologic image comprising: 
a computer; 

a digital memory means accessible to said computer; 
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means for receiving and storing in said digital memory means, an input two- 
dimensional array of digital pixels representing an image produced by radiologic imaging 
apparatus; 

wherein said computer is configured for processing the stored input two- 
5 dimensional array of digital pixels by thresholding at least a portion of the stored input array 
at, at least, 20 different threshold levels to discriminate spots, for each spot discriminated, 
forming a plurality of measures of said spot, including of its size and shape, and determining 
whether the formed plurality of measures fall within predetermined criteria, and producing an 
output two-dimensional array of digital pixels from the input array in which any spots whose 
10 formed measures meet said criteria at any threshold level are marked or enhanced, 

independently of whether or not corresponding spots meet said criteria at more than one 
threshold level, and producing an output two-dimensional array of digital pixels from the 
input array in which any spots whose formed measures meet said criteria are marked or 
enhanced. 

15 8 # The system as claimed in Claim 7, wherein said 

formed measures include one of the variance or uniformity of values of the digital pixels 
comprising said spot. 
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